Digital Globe Fusion Contest

The IEEE GRSS is running its annual Data Fusion Contest featuring a WordView2^® multispectral satellite data set supplied by Digital Globe. The contest has been run since 2006 and is designed to promote the use of such high-resolution multispectral data. It also serves to advance the art and science of extracting information from satellite imagery. This year the Data Fusion Contest aims at exploiting multi-angular acquisitions over the same target area using the Digital Globe 8-band WorldView2^® multispectral product. Contestants are asked to identify a problem and then use the data set provided by Digital Globe to solve the problem. The contestants then summarize their work in a short paper and submit it for judgment.

PANCROMA v4.07 is able to read and process 8-band WorldView2^® data as commonly packaged by Digital Globe. You can Download a free trial copy for experimenting on this data set. The contest provides an interesting platform for exploring and testing the capabilities of the latest PANCROMA version on 8-band Digital Globe WorldView2^® data. The application works very well for Landsat, SPOT^® and ASTER. I wanted to test it using the provided WorldView2^® data sets.

To download the data, you simply register on the Digital Globe Contest Site and download the zip file. After extracting, you will have a file folder entitled "RIO_sequence_DFC" under which will be two subfolders. One subfolder (052333524020_01) holds the multispectral data and the other (052333524010_01) holds the corresponding panchromatic bands. There are five data sets in all. They cover the same area of the Rio de Janeiro harbor area except that the satellite acquisition angle is different for each data set. Metadata for each set is contained in the .IMI and XML files in the multispectral folder. The resolution of the multispectral bands is 2m per pixel. The resolution of the panchromatic bands are 0.50m/pixel, so the panchromatic band files are 4X the multispectral bands. Well, almost. More on this later.

The first step in processing this data using PANCROMA is to set up a file structure. I set up five folders for each multispectral data set and five for each of the panchromatic bands. The file structure is necessary for two reasons. The multispectral data is organized in planarConfiguration=2 GeoTiff files. This means that the 8 multispectral bands are saved into a single GeoTiff file. The TIFF file standard allows saving files this way as one option. PANCROMA wants to extract the band data and save the 8 bands as individual GeoTiff files. Thus the five folders to keep everything organized.

The second reason is that the panchromatic bands have an 11 bit dynamic range. The 11 bits of pixel brightness are stored as 16 bit (GeoTiff WORD) data types. In order to view or to use the panchromatic band for pan sharpening, the 11 bits must be resolved into an 8-bit RGB color model (unless you have the special hardware and software for displaying the data in a higher bit-depth color model). PANCROMA offers three methods for doing this. The first method is to scale each pixel by a ratio of 2⁸/MAX_VALUE where MAX_VALUE is the largest grayscale level in the image. This is the default PANCROMA method. Sometimes however a relatively minor number of pixels have MAX_VALUE = 2¹¹-1 = 2047 resulting in a dark image. In this case, PANCROMA allows the user to bit-shift each pixel value by up to three. (Each bit shift is equivalent to dividing the pixel value by 2). Although a small number of pixels may saturate, this may yield better results in some cases. A third method is to ratio the pixels by 2⁸/MAX_VALUE and then subject the grayscale image to a histogram equalization or stretch. This can greatly improve the resulting pan sharpened image contrast. However histogram operates can affect color fidelity for the pan sharpened image as well, so this should be done carefully if creating true color images.

To extract the multispectral band data, select 'File'|'Open' and open the first multispectral bundle TIF file. Now select 'Display One File'. The 'Decompose Eight File Digital Globe Bundle' menu selection will be enabled. Select 8-bit format. Once you select it, you will be prompted via a Save Dialog box to enter an output file name. You must provide a base file name to which a numerical suffix will be added to, for example 'outputfile' which will become 'outputfile-1.tif', outputfile-2.tif, etc. When you accept the file name, PANCROMA will decode the bundle file and write individual GeoTiff band files to the directory that you designated. The numbering convention will be consistent with that for Landsat band data, to an extent practical. That is, band1 will be the shortest wavelength (ocean blue) band, band2 the next longer (blue), band 3 the next longer, etc. Consult the Digital Globe web site for band file specifications. (The blue, green and red bands are band2, band3 and band4, respectively.) Do this for each of the TIFF bundle files until you have five sets of 8 band multispectral data. (After each operation reset PANCROMA by selecting 'Close Graphic Window and Reset'.)

Now you can process the panchromatic bands. Select 'File'|'Open' again and open the first panchromatic band. elect 'Display One File'|'Display One Grayscale Image'. (Note: I simply accepted the default (2⁸/MAX_VALUE) method which produced an acceptable panchromatic image.) You must save it as an 8-bit grayscale image by selecting 'File'|'Save Grayscale Image'. Save it in GeoTiff format to preserve the embedded georeferencing data. Select 'Close Graphic Window and Reset' and repeat the operation for each panchromatic image. Note that it is important to open the panchromatic bands and then save them as GeoTiffs in order to convert them to 8-bit format. If you try to process them in 16-bit format you will get strange images.

You should now have 5X8=40 extracted multispectral band files in their respective folders and five corresponding 8-bit panchromatic bands. Notice that each multispectral band has 1000 rows and 1001 columns while the panchromatic bands each have 4000 rows and 4000 columns. PANCROMA wants the panchromatic bands to be even multiples of the multispectral band sizes in order to pan sharpen, so it may be convenient to resize the extracted multispectral band files down to exactly 1000 rows by 1000 columns now. To do this select 'Close Graphic Window and Reset' again and then open the first four multispectral bands in one of the five data sets. Now select 'Pre Process'|'Resize Images'|'Resize Four Images'. A Resize Data Input screen will become visible. The number of rows and columns will appear in the text boxes. Change the column value from '1001' to '1000' and select 'OK'. The four files will be resampled down to the target row and column sizes. Select 'Close Graphic Window and Reset' and repeat for the other four band files in the set, then repeat for the other four data sets.

Although the files are not very large, I found it convenient to subset the sets for faster iterating as I determined my optimal settings to produce the images that I wanted. I was interested in the blue, green and red bands plus the panchromatic band and the NIR band. So I subsetted the resized (to (1000X1000) bands 2, 3, 5, panchromatic (4000X4000) and band 7 (1000X1000). In order to do this, select 'Close Graphic Window and Reset' and then open the files in the listed order. Then select 'Band Combination'|'Subset Images'|'Subset Five Bands'. You can use three methods to identify the corner coordinates of the subset files: either by clicking on the image, by selecting the 'Rubber Band Box' button, or by choosing the 'Select by Coordinates' radio button and keying in latitude and longitude coordinates. Once you identify your corner coordinates select 'Accept Pixel Entries' if enabled and then 'Enter'. Then save your files by selecting 'File'|'Save Subset Images'|'GeoTiff'. (The subset images will become visible upon saving.)

It is always helpful to create a simple RGB natural color image from the subset blue, green and red bands 2, 3 and 5. I started with the bands extracted from file 10JAN19130923-M2AS-052333524020_01_P001.TIF. Start by opening the files in the stated order and select 'Band Combination'|'Generate RGB True Color Composite'. My first subset RGB image is shown to the right. Note that the color tones are not very natural looking, with too much green tone. I attempted to improve the outcome using PANCROMA image processing utilities without much success.

I received feedback from Fabio Pacifici of Digital Globe on this issue. Fabio pointed out that in contrast to Landsat WorldView2, QuickBird and other very high resolution satellites can acquire images at high off nadir angles. This presents a much longer atmospheric path to the satellite sensors, and can be one of the reasons of the much blue-green coloring. He recommended calibrating the images for better color results.

The calibration procedure is described in the Digital Globe publication entitled Radiometric Use of WorldView-2 Imagery available at the Digital Globe website. Quoting from this publication: "WorldView-2 products are delivered to the customer as radiometrically corrected image pixels. Their values are a function of how much spectral radiance enters the telescope aperture and the instrument conversion of that radiation into a digital signal. That signal depends on the spectral transmission of the telescope and MS filters, the throughput of the telescope, the spectral quantum efficiency of the detectors, and the analog to digital conversion. Therefore, image pixel data are unique to WorldView-2 and should not be directly compared to imagery from other sensors in a radiometric/spectral sense. Instead, image pixels should be converted to top-of-atmosphere spectral radiance at a minimum."

Calibration requires multiplication of each grayscale image digital number (DN) by the Absolute Calibration Factor and then dividing it by the Effective Bandwidth. After calibration, each of the grayscale bands must be rescaled to reasonable brightness levels using the same scale factor for each image. I wrote the code required for the calibration and scaling and incorporated it into the latest version of PANCROMA. To calibrate an image you must first obtain the Absolute Calibration Factor (absCalFactor) and Effective Bandwidth (effectiveBandwidth) from the .IMI file that accompanies each data set. Note that each of the eight bands has its own calibration coefficients. Each band must be calibrated using its own calibration coefficients but scaled using a common scale factor.

To calibrate, open a single grayscale image using the 'File|Open' menu selection. After opening the file, select 'Pre Process'|'Calibrate WorldView2 Band'. A data input form will be displayed. Insert the calibration coefficients for the band you are calibrating into the text boxes on the data input form. You can use the default scale factor, or input one of your own. The optimum scale factor can be computed as:

Scale Factor = SUM(absScaleFactor)/SUM(effectiveBandwidth)

Where

SUM(absScaleFactor)= (absCalFactor1 + absCalFactor2 + absCalFactor3)

And

SUM(effectiveBandwidth = (effectiveBandwidth1 + effectiveBandwidth2 + effectiveBandwidth3)

After calibrating the band and panchromatic images using the default scale factor of '4', I saved them to disk. I then applied a proportional histogram stretch to each of the bands in turn and saved them again. (Note: it is not necessary to stretch the panchromatic band as it will be automatically histogram matched during the pan sharpening operation.)

Histogram operations are an art and science onto themselves, and the subject merits another entire article. Histogram operations can improve satellite image quality significantly, but they can also render them into an unusable mess. I used a proprietary PANCROMA algorithm that stretches the pixel classes at the edges more than those closer to the mean. You may find other algorithms that work equally well or better.

I then selected 'Close Graphic Window and Reset' and then reloaded the processed subset bands 1, 2 and 3 (blue, green and red). I created an RGB composite image, using a bit of haze reduction and contrast enhancement. The result is the second image on the right. The resulting image is much better with much more realistic colors.

You can now try pan sharpening the image. Select 'Close Graphic Window and Reset' and then open subset bands 1, 2, 3 (blue, green and red) and then the panchromatic subset band 4. Now choose 'Pan Sharpeneing'|'HSI Transform'|'Standard Four File'. A data entry box will appear. I selected 100 iterations for the HSI Interpolation Factor and clicked 'OK'. After a moment the pan sharpened image appeared. If you run this yourself you will see the histogram data box appear during the run. PANCROMA automatically matches the histogram of the panchromatic band to the computed intensity band as its default condition. The resulting image appears to the right.

My first pan sharpening attempt had a lack of absolute registration among my expanded hue, saturation and panchromatic images. I remedied this by shifting the hue and saturation images two pixels to the south. In order to do this, I checked the 'Array Offsets' check box on the Image Processing Data Input Form and then selected the number of pixels to index from the drop down selections. Positive numbers shift the image south and west for row and column adjustment and vice-versa for negative numbers. I entered '2' in the 'Row' box and left the columns alone. This brought the images into reasonable alignment.

I tried a second run adding Thin Plate iterations to the image interpolation PANCROMA normally uses a simple but very fast Laplacian interpolation algorithm. The Laplacian is also convenient because it converges rapidly. The Laplacian is not very efficient at extracting information from beyond its nearest neighbor pixels. This is not very important for interpolating Landsat or SPOT^® data where the panchromatic band is 2X the multispectral bands. 4X data like WorldView2^® presents a sparser interpolation. PANCROMA can run a Thin Plate preprocessing interpolation algorithm if you instruct it to. The Thin Plate algorithm is also very fast and it reaches out beyond nearest neighbor pixels to interpolate. One disadvantage is that it may ring at higher frequency areas of an image. That is why it is better as a preprocessing step using relatively few iterations than as a primary interpolation method.

To add Thin Plate iterations, check the 'Thin Plate Check Box' and select your desired iterations from the drop down Combo Box. These will be added to the Laplacian iterations that you specified in the pan sharpening data input box. My results after making these two adjustments can be seen to the right.

I tried a pan sharpening run using five file XIONG processing. This technique uses the NIR band in addition to the blue, green and red bands plus panchromatic band. Using the information in the NIR band can improve the pan sharpened image for file sets where the panchromatic band overlaps the NIR band significantly. The WorldView2 panchromatic band overlaps the NIR band less than Landsat does, for example. My experiments in this regard showed that the slight improvement in green tones that resulted from XIONG processing did not outweigh the negative artifacts that are introduced when using this technique for 4X interpolated images and is not particularly recommended for WorldView data.

Finally, I applied the information that I learned processing the subset images to the full sized image. The 4000X4000 pan sharpened image is shown to the right. I took a screen shot of my settings in case you want to reproduce my work, and use it as a starting point for your own.

Writing this article provided an excellent opportunity to test PANCROMA's capabilities against WorldView2^® multispectral and panchromatic data. Although PANCROMA performs very well when processing Landsat and SPOT^® data, WorldView2^® presented some new challenges and highlights some opportunities for improvement of the program. In fact I added a few new features as I worked through the data and developed some definite ideas for additional capability.

This is just a brief survey of the data set. Undoubtedly better results could be achieved with more in-depth work. There are also many additional operations that you can experiment with on this data set, including using unsupervised classification, band arithmetic, histogram operations, as well as using PANCROMA's broad selection of image processing utilities. Information on these, as well as instructions, tutorials and Videos are available in the Instruction Manual. I hope this article will get you started investigating the potential of multispectral data sets. All that you need now is some creative thinking and hard work. Good luck if you plan to enter the contest!

Thanks to Fabio Pacifici for his very helpful corrections and comments to this article.